feat(extensions): let extensions contribute always-on instructions by TheovanKraay · Pull Request #4259 · github/spec-kit

TheovanKraay · 2026-08-21T17:59:35Z

Description

Part of #4200.

Adds a provides.instructions capability so a Spec Kit extension can ship a compact,
always-on rule block that reaches the agent without any command or hook being invoked —
the reachability gap discussed in #4200. Ownership follows what we agreed on in the thread:

core only validates the new metadata (type + path-safety) and never writes agent files.
the opt-in agent-context extension composes enabled extensions' blocks into the routed
agent file (.github/copilot-instructions.md, CLAUDE.md, …) as namespaced
 blocks, and owns routing, refresh, update, and removal.
if agent-context isn't installed, nothing touches any agent file. The key is additive and
a no-op on cores without this change.

Why it's needed: today nothing delivers an extension's guidance automatically on install in a way
that survives a hands-off agent run. This adds that always-on channel, scoped to an opt-in extension.

Efficacy (same rule block + conformance metric throughout):

Isolation pre-check, n=24 (2 models × 4 langs × 3 complexity): A bare 0.843, B always-on 0.986,
C rules in an invoked command 0.989, D same command never invoked 0.831. C − B = +0.003,
payload present for B/C and absent for D — so the effect is reachability (present vs absent), not
obedience.
Magnitude, bare vs the exact block this mechanism composes, n=24: +0.123 mean conformance,
22 wins / 0 ties / 2 losses. The block that lands in copilot-instructions.md on a real install
is byte-identical to the one measured.

One test not completed (for transparency): a behavioral A/B on an internal at-scale agent-execution
harness is currently inconclusive due to an agent-runtime regression unrelated to this change — the
agent crashed on startup on every attempt (both arms, incl. bare), confirmed by an untouched baseline
image failing identically. In-container delivery of the block was verified separately. That datapoint
is pending a healthy agent build and is not reported as a result.

Testing

Tested locally with uv run specify --help
Ran existing tests with uv sync && uv run pytest
Tested with a sample project (if applicable)

New tests tests/extensions/test_extension_instructions.py: 13 passed (validation accept/reject,
composition, disable-cleanup, multi-extension ordering, path-unsafe skip, no-op without agent-context,
emit mode). Full suite on this branch, rebased on current main: 6916 passed, 415 skipped, 0 failed.
bash / powershell / python emit paths verified byte-identical. End-to-end verified in a sample
project: after specify extension add <ext> --dev + specify extension add /agent-context --dev, the
namespaced block appears in .github/copilot-instructions.md, and enable/disable/remove clean up correctly.

AI Disclosure

I did not use AI assistance for this contribution
I did use AI assistance (describe below)

Implemented with GitHub Copilot (agentic) — the core validation, the agent-context compose logic and
its bash/powershell/python twins, and the test suite were written with AI assistance and reviewed by me.
The empirical evaluation (the pre-check and the +0.123 conformance measurement) was also run and analyzed
with AI assistance.

…ithub#4200) Adds a provides.instructions capability so an extension can ship a compact always-on rule block that reaches the agent without any command/hook invocation. Ownership per maintainer decision: core validates the metadata only; the opt-in agent-context extension composes and owns the agent-file writes (namespaced  blocks, per-agent routing, enable/disable/remove lifecycle). No agent-file writes when agent-context is not installed. core: accept+validate provides.instructions (path-safe, instructions-only extension allowed), expose .instructions. agent-context: compose enabled extensions' instruction blocks into the routed context file; bash/ps1 twins delegate to the python twin's --emit-extension-blocks for byte-identical output. Tests: tests/extensions/test_extension_instructions.py (13). Evidence: extensions/agent-context/INSTRUCTIONS-POC-EVIDENCE.md.

Copilot

Pull request overview

Adds opt-in, always-on extension instructions composed through agent-context.

Changes:

Validates provides.instructions manifests.
Composes enabled extensions’ instruction blocks across script variants.
Adds validation/composition tests and supporting evidence.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
`src/specify_cli/extensions/__init__.py`	Adds manifest validation and accessor.
`extensions/agent-context/scripts/python/update_agent_context.py`	Collects and renders instruction blocks.
`extensions/agent-context/scripts/bash/update-agent-context.sh`	Delegates block rendering to Python.
`extensions/agent-context/scripts/powershell/update-agent-context.ps1`	Delegates block rendering to Python.
`tests/extensions/test_extension_instructions.py`	Tests validation and composition.
`extensions/agent-context/INSTRUCTIONS-POC-EVIDENCE.md`	Documents the prototype and evidence.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

…on payloads; ps1 PyYAML probe; docs + evidence Copilot PR review (github#4259) fixes: - update_agent_context.py: reject instruction payloads that embed a managed-section marker (outer markers or per-extension SPECKIT EXT markers) so _upsert_section cannot strand content on disable/remove; catch UnicodeDecodeError (not just OSError) so a non-UTF-8 file is skipped instead of crashing the refresh. Markers threaded through the collector/render helpers. - update-agent-context.ps1: the emit-blocks interpreter probe now requires 'import yaml' (mirrors the config-parse probe) so a python3 without PyYAML is not selected. - EXTENSION-API-REFERENCE.md + EXTENSION-DEVELOPMENT-GUIDE.md: document provides.instructions (schema, path rules, opt-in agent-context behavior + lifecycle). - INSTRUCTIONS-POC-EVIDENCE.md: use the verified +0.123 (22/0/2, n=24) install-path figure and mark the earlier +0.142 as a distinct pilot; refresh suite counts; clarify the trigger/lifecycle model. - tests: add marker-collision-skip and non-UTF-8-skip cases (15 passing).

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Suppressed comments (1)

extensions/agent-context/scripts/python/update_agent_context.py:493

This composition only runs when the update script is invoked. The extension add/remove/enable/disable paths only refresh native events, and agent-context registers only after_specify/after_plan hooks, so adding an instructions extension leaves the context file unchanged until a user or agent invokes a command/hook; disable/remove likewise leaves stale rules until then. That contradicts the PR's “without any command or hook being invoked” guarantee and #4200's install/lifecycle acceptance criteria. Add an agent-context-owned extension-lifecycle refresh and cover add, enable, disable, remove, and update end to end.

    extension_blocks = _render_extension_block_lines(project_root, marker_start, marker_end)
    section = _build_section(marker_start, marker_end, plan_path, extension_blocks)

…t parity test for composed instructions - __init__.py: reject a present non-string provides.instructions[].description, matching the docs and the command/template/script validators. - test_update_agent_context_python_parity.py: add an installed-instructions fixture and compare Python/Bash/PowerShell resulting context bytes incl. a non-ASCII payload (bash gated to POSIX CI; PowerShell parity verified locally). - test_extension_instructions.py: add non-string-description validation test.

Copilot

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

extensions/agent-context/INSTRUCTIONS-POC-EVIDENCE.md:58

This count is stale: the file contains 14 test functions, and the three-case parametrization produces 16 collected test cases.

`tests/extensions/test_extension_instructions.py` (13 tests, all passing):

…PowerShell has no Python for composition - emitter (--emit-extension-blocks) now accepts --marker-start/--marker-end; bash and ps1 twins forward their configured markers so collision-rejection uses the SAME markers the upsert uses (fixes custom-marker payload collisions, not just default SPECKIT markers). - ps1: when no Python 3 + PyYAML is on PATH but other extensions are installed, warn that provides.instructions blocks were not composed instead of silently writing only the base section (bash already requires Python for its upsert, so it cannot silently omit). - tests: custom-marker forwarding + collision-rejection unit tests; custom-marker byte-parity tests for bash (POSIX CI) and PowerShell (passes locally).

Copilot

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Copilot AI balanced review requested due to automatic review settings August 21, 2026 17:59

TheovanKraay requested a review from mnriem as a code owner August 21, 2026 17:59

Copilot started reviewing on behalf of TheovanKraay August 21, 2026 18:00 View session

TheovanKraay changed the title ~~feat(extensions): let extensions contribute always-on instructions (#…~~ feat(extensions): let extensions contribute always-on instructions Aug 21, 2026

Copilot AI reviewed Aug 21, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 24, 2026 11:56

Copilot started reviewing on behalf of TheovanKraay August 24, 2026 11:57 View session

Copilot AI reviewed Aug 24, 2026

View reviewed changes

Comment thread src/specify_cli/extensions/__init__.py

Comment thread tests/extensions/test_extension_instructions.py

Copilot AI review requested due to automatic review settings August 24, 2026 14:11

Copilot started reviewing on behalf of TheovanKraay August 24, 2026 14:11 View session

Copilot AI reviewed Aug 24, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 24, 2026 15:25

Copilot started reviewing on behalf of TheovanKraay August 24, 2026 15:26 View session

Copilot AI reviewed Aug 24, 2026

View reviewed changes

Navigation Menu

Uh oh!

feat(extensions): let extensions contribute always-on instructions - #4259

feat(extensions): let extensions contribute always-on instructions#4259
TheovanKraay wants to merge 4 commits into
github:mainfrom
TheovanKraay:feat/extension-instructions

TheovanKraay commented Aug 21, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

TheovanKraay commented Aug 21, 2026

Description

Testing

AI Disclosure

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants